Multi-Resolution Video Streaming in 
Peer-to-Peer Networks 

Batuhan Karagoz*, Semih Yavuz^, Tracey Ho*, and Michelle Effros* 

* Department of Computer Engineering, Middle East Technical University, Ankara 06800, Turkey 

'Department of Mathematics, Bilkent University, Ankara 06800, Turkey 

^Department of Electrical Engineering, California Institute of Technology, Pasadena, California 91125, USA 

batu@ceng.metu.edu.tr, y_semih@ug. bilkent .edu.tr, t ho Seal tech . edu, 
~Pf\~- effros@caltech.edu 

^— v ' Abstract 

We consider multi-resolution streaming in fully-connected peer-to-peer networks, where transmission rates are 
^V constrained by arbitrarily specified upload capacities of the source and peers. We fully characterize the capacity region 

of rate vectors achievable with arbitrary coding, where an achievable rate vector describes a vector of throughputs 
of the different resolutions that can be supported by the network. We then prove that all rate vectors in the capacity 
region can be achieved using pure routing strategies. This shows that coding has no capacity advantage over routing 
in this scenario. 
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I. Introduction 



We consider multi-resolution streaming in a heterogeneous peer-to-peer setting, where peers have different upload 
capacities and demand an information stream at different resolutions. The information stream is layered, such as in 
Scalable Video Coding |T), which generates a base video layer and a number of enhancement layers that depend 
on the base layer and all lower layers. 

We assume a fully-connected overlay network in which transmission rates are constrained by the upload capacity 
of the source and each peer, a model introduced in [|2] to capture the most important constraints in peer-to-peer 
ry-\ \ networks. A problem instance is defined by specifying the number of layers demanded by each peer and the upload 

l/~) ■ capacity constraints of the source and each peer. Our goal is to find the capacity region of achievable rate vectors, 

where an achievable rate vector describes a vector of throughputs of the different resolutions that can be supported 
by the network. 

Solutions can be classified as follows. Inter-session coding solutions are the most general, allowing coding across 
information from different sessions (i.e. layers). Intra-session coding solutions restrict coding to occur only within 
each session. Routing solutions allow only replication and forwarding of information at each node. Intra-session 
coding corresponds to independent multicast network coding for each layer, for which the capacity region is given 
by a linear program. In contrast, characterizing inter-ses sion coding capacity, which corresponds to the information 
theoretic capacity, is open for general networks. 

Related work by Chiu et al. [3] studies the special case of a single resolution. That case corresponds to a single 
multicast, and (3) shows that network coding is not needed to achieve capacity. In [4|, Ponec et al. consider the multi- 
resolution case restricted to intra-session coding, showing that intra-session coding does not improve the capacity 
region over routing. A different objective of minimizing average finish times for file download was studied in 0, 

a 

In this paper we provide a complete characterization of the capacity region of feasible rate vectors achievable 
with arbitrary (inter or intra-session) coding, and show that the entire capacity region can be achieved with routing. 

II. Problem Definition 

A peer to peer network is modeled as a complete directed graph with a single source node po and k > 1 peer 
nodes {po,pi, . . . ,pk}- The upload capacities of nodes po, . . . ,Pk are Co, . . . , Ck respectively. 

Information originates at the source node and is distributed to the peers, which help the distribution process by 
uploading information to other peers. Coding may occur at the source and peers. 

Let 77 be the number of different resolutions in a layered data stream. We denote by x±, . . . , x n the data streams 
corresponding to the different layers, such that the jth resolution corresponds to {x±, X2, ■ ■ ■ ,Xj}. The rate of Xj is 
denoted by Lj. For simplicity, we assume that the upload capacities Cq, . . . ,C\ and the data rates L\, . . . , L n are 
integers, which can be approached arbitrarily closely by scaling the unit appropriately. 



We are given nested sets Xi,. . ., X n specifying the demands: 

Xj = {pi\pi demands xf\. 

We also define X n+ i = {po}, so we have 

{pa} = X n+1 CX n C...CX 1 = {p ,Pi, ■ ■ ■ ,Pk}- 

For all Si, S2 Q X±, Si — > S2 is defined as the set of all links coming from Si and going to S2. We also write 
Pi — > Pj instead of {p t } —} {pj} for brevity. The constraints on a successful transmission scheme are as follows: 
i) Each outgoing link of the source p is a function of Xi, . . . , x n : 

Vpi e Xi, H(p -t-pi\xi,...,x n ) = 0. 

ii) Each outgoing link of pi G Xi \ {po} is a function of incoming links: 

Vpi,Pj €Xi\ {po}, H(p, -)■ pj\p -)■ ^, . . . ,p„ ->• pj) = 0. 

We assume that H(pi — > pi) = without loss of optimality. 
Hi) Each peer pi G Xi can transmit at rate at most Cf. 

iv) Each peer pi G X_, \ {po} is able to decode Xj from its received information: 

I(Xi -+Pi-,Xj) = ^(xj) = Lj. 

III. Approach 

In this section we provide some intuition for our approach. A first observation is that total upload capacity should 
be greater than the total rate of data which has to be delivered: 



J2Ci>J2\ X i\ L i- C 1 ) 



i=0 i=l 

This condition is necessary but not sufficient. The following sequence of lemmas leads to a sufficient condition for 
a rate vector to be achievable. The proofs of lemmas in this and the next section can be found in the Appendix. 

Lemma 1. Let k and Co be positive integers and Ci, C2, . . . , Cfc be nonnegative integers such that 

k 

c a +Y. c ^ k 

Then there exists a directed tree rooted at Vq with vertices Vi, . . . ,Vk such that 

outdeg(vo) = C , 

Vi G {1, . . . , k} outdeg{vi) = Ci, indeg(vi) = 1. 

Lemma 2. Data x with rate L can be transmitted to peers pi , . . . , p^ by using source capacity Cq and peer upload 
capacities Ci , . . . , Cfc if 

Cq > L and 

k 
Y^ C i> kL - 

4=0 

Lemma 3. Given the sets of peers Xi,X%,...,X n and upload capacities Cq,Ci, . . .,Cfe, the rate vector 
(Li, L2, . ■ ■ , L„) is achievable if for every j G {1, . . . , n} 

j-l n 

Y, Ci>j2 L i+J2\ x i\ L i ( 2 ) 

Pi^Xj i—l i—j 



and 

n 

Co>Y, L - ( 3 ) 

Intuitively, if one of the inequalities in Lemma [3j say the jth one, does not hold, this means that the nodes in 
set Xj cannot handle the transmission of data layers Xj through x n . Hence, some peers from the set X\ \ Xj need 
to help in transmitting those data layers, necessitating some additional capacity for transmitting this data to peers 
in X\ \ Xj which do not themselves demand it. This requires additional capacity beyond that given in (fl}. 

To characterize this explicitly, it is useful to define the margin of the jth inequality: 

n j—1 

n, = y \Xi\u +Y L >- J2 °i- 

i—j i—1 piGXj 

For completeness we also define the (n + l)-th margin N n+ i as zero. The capacity region derived in the next section 
is stated in terms of these margins. In fact, not all of them, but a special subset of them, will be used. This subset 
is defined as follows: 

Definition 1. For a finite sequence {a n } = a\,...,a 8 , the dominant subsequence of {a n } is the subsequence 
{a in } = a h , . . . , a ih defined by 

i) i h = s 

ii) ij is the greatest index such that ij < ij+i and Oj. > a% j+x . 

IV. Converse Bound for Capacity Region 

In this section, we present a converse bound on the capacity region, which is shown to be tight in the following 
section. 

Theorem 1. Given the sets of peers Xi,X%, . . . ,X n and upload capacities Co,Ci, . . . ,Ck, if the rate vector 
(L\, L,2, . ■ ■ , L n ) is achievable by any coding scheme, then 

pi£Xi i=\ 1=1 ' *' 

where N^ , ■ ■ ■ , Nd h+1 is the dominant subsequence of Ni, . . . , N n+ i. 

Proof: For a resolution Xj and a peer pi G Xj we have, from property iv in Section HI] 

H(xj) = I(Xi -)■ pi-Xj) = I{Xi \ X 3 -)■ p h Xj ->■ p 4 ; Xj). 

If we view X\ \ Xj as a supernode, outgoing links should be functions of incoming links, since peers in the set 
Xi \ Xj do not create additional data besides incoming data (property ii). Hence, links in set Xi \ Xj — > pt are 
completely dependent on links in set Xj — > X i \ Xj. Then, we may write: 

H(xj) = I{{X 1 \Xj)^ p u Xj -»• p^xj) < 

I{Xj -> {X x \ X^^Xj -> Pl - X j) < H{ Xj ) 

=► H(X j )=I(X j ^(X 1 \X j );x j ) 

+I(X j ^p i ;x j \X j ^(X 1 \Xj)), 

Summing this for all peers in Xj yields 

\Xj\H(Xj) = \Xj\I(Xj -> (X 1 \X j );x j )+ 

Y, i(x^P.;i 3 ft^(ii\i;)). 

p.eXi 



By rearranging this, we can obtain 

I(Xj -+ (X, \ Xj)- Xj ) = — f- [\X 3 \H{ X j)- 

\Aj\ - 1 



1 



I(X j ^(X l \X j );x j y 



J2 I{*3 -> PiiXiM -► (X x \ Xj))}. (4) 

The left hand side of this equation can be replaced by parameters which are independent from the transmission 
scheme by using the following lemma: 

n 

Lemma 4. £ I(X j -► (X x \ X,-);*,-) < £ pi£X . ^ - E"=i l*;l# (*;)■ 
i=i 

By using Equation © and Lemma [U we can obtain: 

n n 1 

E q > E \ x i\ H &i) + E ijfTiiB^i%) 



PiGX, 



Now define A,- as 



A j = ^2\X l \H(x l )-J2[I(Xi^(X l \X l );x l )+ 

1=3 1=3 

X J(*j->ft;a:j|Xi->(*i\*i))]- 

Piext 

For the sake of completeness, define also A n+ \ = 0. Putting this into Inequality (01, we obtain: 

n n A A 

E^E^i^+Ei^f- (6) 

We also have, from Equation (0): 

i4j - A,- +1 = (I*, I - l)/(-X-j ->■ (Xj \ X,); »j) > 0. (7) 

Note that the A,- values are determined by the transmission scheme. To obtain a bound which is independent 
from transmission scheme, we use the following lemma: 

Lemma 5. Aj > Nj. 

Let us examine the last sum in ([6j: 

E A 3 - A? + l _ V^ A 3 ~ A 3 + l , V^ A 3 ~ A 3+l , 
|X,|- 1 " ^ IX.-I-1 + ^ 



j=l J j=l J j=di 

A 3 - A 3 + l 



XA-l 



E 



\Xh 

3=d h ' J 



Using inequality (0, 



d 2 -l a a n 

u+ 2-, x dl -l .^ 

4fa , A ( 1 



4,--; 


y+i 


\x dh \ 


- 1 







|X dl |-l ^\\X d2 \-l \X dl \-l 

1 1 



> J^ + nJ l 



x dl \~i ^\\x d2 \-\ \x dl \-i 
i i 



N, 



h \\x dh \-i \x dh ^\-i 

N di - N di+1 



S 1**1 "1 ■ 

The last inequality is due to Lemma [5] . By putting this result in (O, we obtain: 



Pi£X x i=l 



U i^i- 1 



V. A Routing Scheme that Achieves the Capacity Region 
In this section, we give a transmission scheme using multicast routing trees that achieves the bound in Theorem 

m 

Theorem 2. Given the sets of peers X\,Xz, . . . ,X n and upload capacities Co, C±, . . . , Cf., the rate vector 
{L\,Lii . . . , L n ) is achievable by routing if 



C >J2 L i> ( g ) 



h 



E^Ei^+E^rf (9) 

Pi£X± I— 1 I— 1 ' 

where N dl , . . . , N dh+1 is the dominant subsequence of N\, . . . , N n+ i. 

Proof: The proof is by induction on the number of inequalities from Lemma [3] which are not satisfied. For this 
purpose let us define the set 

I={t\N l >0}. 

We will show that we can reduce the size of / by at least one, by using some amount of capacity, such that the 
residual system also satisfies (0 and (0. 

The base case is the one where / is the empty set, i.e. all Nj values are less than or equal to zero and it is 
examined in Lemma [5] Let m and M denote the minimum and maximum elements of /, respectively. The following 
lemma is essential for our method: 

Lemma 6. Vp^ e X\ \ X m , 3C\m with < CW < C\ such that 

Y, C iM <N-N 3 
Pl eXj\x m 

Vj G {1, 2, . . . , m — 1} and 

/ J ^iM — 



\x M \ - 1 

Pi£Xi\X m ' Ml 



where N = min(N m ,Nnj). 



Now, for each pi 6li\ X m let us choose Cm as in Lemma [6] Then we have 

\Xm\N 
Xm|-1 



v \Xm\N 



P.eXi\x m ' 

where JV = miri(iV m ,iVjv.r)- Let us also define 

TV 



Com = 



|^m| - 1 
Now take a rate- , X N , 1 portion of xm, called s. Then, using equality ( fTOb . the rate of s is given by 

N m—^ CiM 

PifcAl \A m 

Hence we can divide s into \X\ \ X m \ portions s, corresponding to peers p, G X\ \ X m , where the rate of portion 
Si is given by £ iM , . To each peer p^ Gli\ X m , send s, from the source. This consumes Com amount of capacity 
of the source. Then, from each pi 6 X\ \ X m , send Si to the peers in Xm- This consumes CW amount of capacity 
of each pi S X\ \ X m . In this way, transmission of portion s is completed. After the procedure, we have residual 
capacities 

( Ci - Cm i/ p, eX x \X m 
C' t = l C i/ P,el m \ {p } 

{ C o-^pr ifPi=Po- 

and residual data rates 

1 \ Li otherwise. 

The Ni values are updated accordingly. Denoting the updated value of Nj by TV'-, we calculate it differently for 
three cases: 
i) If j < m: 

n j— 1 

i—j ' ' ' ' z— 1 

— 2_^ Ci + 2^ Cjm 

p«eXi\{p } p i ex J \A- m 

= (^LY^ + ^L;- ^ C.) 

i=j i=l Pi<£Xj 

\X M \N N 

'\Xm\ — 1 \Xm 






p ! eA' J \x„ 

= Nj-N+ ]T CiM - 

P ,ex j \x m 

By the choice of CiM values, using Lemma [6) we have 

N$=Nj-N+ J2 c *m<0 (11) 

Pz ex j \x m 



ii) If m < j < M: 



Hi) If j > M: 



n j-1 

n> = y,\ x *\ l '<- c o+Y. l >- E c < 

i=3 »=1 P;6^j\{Po} 

+X>- E ci 

=> iVj = ATj - TV. (12) 

n j—l 

n> = Y,\ x i\ L 'i- c *+12 L 'i- E c 'i 

i=3 »=1 PiGX^\{p } 

n _ J-1 

= Ei^i^-^ + ^-t+E^ 

i— j ' ' z— 1 

* y- c . 

=S> iVj = iVj < 0. (13) 

Now let us examine the dominant subsequence N' d , , . . . , N' d , of N[,...,N^ +1 , Since N„ +1 is zero by 

1 h'-|-l 

definition, N' d , is also zero. Therefore for all i G {1, . . . , h'} 

N' d , > N' d , = =^> m < d'i < M. 

1 h' + 1 

But the order of TV,' values for m < i < M is the same as the order of N values for m < i < M since N = Nj—N. 
Also we know that for all i € {1, . . . , h} 

m<di< M. 

Noting that N dh = Nm, we have two cases: 

(N' d , ..,N' d , ) = (N dl -N,...,N dh - N,0) 

J- h' -\-l 

if N = N m < N M , and 

1 h ' -f- 1 

if TV = AT M < JV m . 

These two cases can be considered as one since even if N is equal to Nm, we can consider as N' d , — = N dh — N 

so that it does not affect inequality (|9j. Hence we can write: 



(N' d , . .,N' . N' d , ) = (N dl -N,...,N dh - N,0). 

1 h ' h -\- 1 



Now let us calculate the left hand side of the inequality (O with updated values: 

^ C[ = C °~ \Xm\-1 + ^ Gl ~ ^ ° lM 
v- „ _ iV |*m|/V 

£ 2.1**^ + 2, ^pi |x M |-i 

Z— 1 2—1 ' l ' ' ' 



1**1 -i l**J-i 

n'„ - r< 

Ei^^ + E 



"- 1 K> - N ' d '+i K' - o 

_^ ^+i + _^ 

1**1-1 \X dh \-l 



i=l i=l ' a '' 

This shows that inequality (0 is preserved after the procedure. Inequality ((HJ is also preserved since 



' ' l— 1 ' ' z — 1 

Now let us look at the updated version I' of /. From (fTTl i. ([T2] | and (fT3] l we know that I' C I . If N = N m , then 
iV,',, = N m - N = ^ m (£ I' ^ \I'\ < \I\ - 1. Similarly if N = N M , then M £ I' =*► |J'| < |7| - 1. Finally 
we can say that after reducing the rate of xm by nr^PT' mec l uant ies ([8]) and (O are still correct and number 
of inequalities from Lemma [3] is reduced by at least one. Hence, by the induction hypothesis, we can complete 
transmission of the remaining data. This completes the proof. ■ 

Combining Theorem Q] and Theorem |2] with the addition of trivial condition Co > X)"=i ^«> we obtain the exact 
capacity region: 

Corollary 1. Given the sets of peers Xi, X2, ■ ■ ■ , X n and upload capacities Co, C\ t . . . , Ck, the rate vector 
(L±, L2, . ■ ■ , L n ) is achievable if and only if the following inequalities hold 



Co>E L « 



(=1 

PiGXi i=\ i=l ' '' 

where N^ , • ■ • , Nd h+1 is the dominant subsequence of N\, . . . , N n+ \. Furthermore, the capacity region is achievable 
using routing. 

VI. Conclusion 

We have characterized the capacity region of achievable rates for multi-resolution streaming in peer-to-peer 
networks with upload capacity constraints, and shown that this region can be achieved by routing. This represents 
a new class of non-multicast network problems for which we have a capacity characterization. Although coding is 
not needed to achieve capacity in this scenario, it can nevertheless be useful in scenarios with losses or without 
centralized control. 



VII. Appendix 



Proof of Lemma 1: 
By Induction on k 
base case: k = Co 



/,■ 



Co + V] C = k = C = outdeg(v ) => ^ C» = 



=1 



Vi G {1, . . . , fc} outdeg(vi) = Ci = 

Then a directed tree rooted at «o with fc leaves, namely Ui,i>2, • • • , Vk, gives us the desired tree. 

Now we can assume that C\ < . . . < Ck-\ < Ck without loss of generality. Then C\ > 1, since it would be 
the base case otherwise. By inductive hypothesis on fc — 1, there exists a directed tree rooted at vq with vertices 

v\, . . . , Vk-i such that 

outdeg(vo) = Co, 

Vi G {1, . . . , k — 2} outdeg(vi) = Ci, indeg(vi) = 1, and 

outdeg(v k -i) = (Ck-i + C k - 1), indeg(v k ^i) = 1 

since 

Co + d + . . . + C fc _ 2 + (Cfc_! + C fc - 1) = fc - 1. 

Now, we add a new vertex, namely Vk, to this tree as an isolated node and then apply the following two changes 
in order: 

1) Remove arbitrarily selected C k outgoing edges from Vk-i where C k > 1, then add a new directed edge from 
Ufc_i to Vk, hence we now have outdeg{vk~\) = Ck-i and indeg(vk) = 1. 

2) Let D be the set of nodes that are disconnected from v^-i in the first step. Note that D is not empty since 
\D\ = Ck > 1. Now add a new directed edge from Vk to each node in set D. Hence, outdeg(vk) = Ck- 

By noting that the number of outgoing/incoming edges from/to the vertices in set D have not been changed, the 
resulting directed tree rooted at vq with vertices v\,V2, ■ ■ ■ ,Vk is exactly the one which is desired. This completes 
the induction and the proof of the lemma. ■ 



Proof of Lemma 2: 
If Co, Ci's or L are not integers we can divide the unit length to least common multiple of denominators to make 
them so. Thus, we will assume that they are integers in this proof. 

We will do induction on L. Base case L = 1 is due to lemma Q] . Now let us assume that the claim is true for 
L — 1. Since 



J2 C > kL, 



i=0 



we can find integers Co, Ci, 



Ck 


such that 










L - 1 < Co 


<C S -1 






o<cT 


<Ci 






0<Cfe 


<c k 






j=0 


b(L-l) 



If we divide the data x into L unit length parts x%, X2, ■ ■ ■ , xl, by the induction hypothesis we can complete the 
distribution of parts X2 through xl by using Co, C\, . . . , Ck capacities. Then we will have capacities Co — Co, Cl — 
Cl , . . . , Cfc — Ck such that 

C -C^>1 



k 

^(C,-Q)>fc. 

i=0 

Hence, we can construct a routing tree for x\ by using the Lemma Q] thereby finishing the distribution of complete 
data x = X\x% . . . xl- ■ 



Proof of Lemma 3: 
We will do induction on the number of resolutions n. 
Base Case : n = 1 is stated in Lemma |2 
Inductive Step: For j = n in (|2), we have 

n-i 
Y, Ci-Y,Li>\X n \L n . (14) 

Pi£X„ 1=1 

Since we have inequalities (0 and ( [Pil l, there exist < C < Cj for each pi 6 X„ \ {po} and L n < Co < 
Cq — Y^i=i Li for source such that 

n-l 

E ^-53i< = |Jf n |L n . (15) 

PiSX„ i=l 

Since we have (TT3T > and Co — X)"==i ■£» — L n , by Lemma [2] each peer in set X n can receive the resolution x n 
completely by using C capacity from each pi £ X„ including source. After completing the transmission of the 
resolution x n , each peer pi <G X n needs to receive only the resolutions xi,X2,- ■ ■ ,x n -\ same as all the peers in 
X n -i- 

If we denote the remaining capacity of a peer p, with C[ and of source with C , we may write 



r > _ j C i_ if p t e X x \X n 

1 1 Ci - a */ pi e x„ 



Then, for each j £ {1, . . . , n — 1}, following is true by using (O and ([T5| >: 

E^-E L * = -E^+ E c 4 +^(c,-cT) 

P.eXj i=i i=i Pi&Xj\x n Pi^x n 

j-i 

= -E L ' + E ^-^nl^n 
i=l P»eXj 

n n—1 

»=J i=3 

Hence we have, for resolutions x\, . . . , x n -i, for every j € {1, . . . , n — 1} 

J-l ra-l 

E a>E^+Ei^i^ 

PiGXj i—1 i—j 

and 

n-l 

a > E L - 

i=l 

Finally we can claim that these capacities are enough for transmission of x\, ■ . ■ , x n ^\ by the inductive hypothesis, 
thereby completing all transmission. ■ 

Proof of Lemma 4: 

For simplicity, we use the notation 

H(S) = H{Xi -> S) 



to represent the entropy of all information X\ — > S recieved by set S and use analogous notation for other information 
theoretic quantities. Then 

n n n j— 1 

Y,i(Xj -> (x 1 \Xj) ]Xj ) kJ^HxAXjw) = EE 7 ^ \x i+ i;x J \Xi+i \Xj) 

j=l j=l j=l i=l 

n — 1 n 

= E E 7 ( x * \ Xi+i ; ^ i Xt+i \ x j ) 

Let us examine the inner sum: 

n n 

Y^ I(X i \X i+1 ;x j \X i+1 \X j ) = H(X i \X i+1 )-H{X i \X i+1 \xi+i)+ E I{X t \X l+1 ;x J \X l+1 \X J ) 

j=i+l ]=t+2 

= H(Xi\X i+1 \xi, . . . ,Xi)+I(Xi\X i+1 ;xi, . . . ,Xi) — H(Xi\X i+1 \xi, . . . ,Xi+i)—I(Xi\X i+1 ;xi, . . . ,Xi\xi+i) 

71 

+ J2 I(X l \X l+1 ;x 1 \X l+1 \X ] ) 

j=i+2 

Since Xi, . . . , Xi are independent from Xi+\ and peers in set Xi \ X + i can decode Xi, . . . , Xi'. 

= H(Xi \X i+1 \xi,...,Xi) +H(xi,...,Xi) - H(X { \X i+ i\xi,...,x i+ i) - H(xi,...,Xi) 

n 

+ Y, HXiXXi+nxjlXi+^Xj) 

j=i+2 

a 

= H(Xi \X i+1 \xi,...,Xi) - H(Xi \X i+ i\xi,...,Xi+i) + Y I ^ Xl \X i+ i;xj\X i+1 \Xj) 

j=i+2 
n 

= H(Xi\X i+1 \ Xl , . . . ,x- l )- J tf(X t \X +1 |x 1 , . . .,x i+1 )+ J2 [H{X i \X w \X i+1 \X j )-H{X i \X i+1 \x j ,X i+1 \X j )] 

j=i+2 

= H{Xi\X i+1 \xi, ...,Xi)- H{Xi \ X i+1 \ Xl , . . .,x i+1 ) + H{Xi \ X i+1 \X i+1 \ X i+2 ) 

n 

— 2_^i [H(Xi \ Xi+i\xj , Xi+i \Xj) — H(Xi \ Xi+i\Xi+i \ Xj+i)] — H(Xi \ Xi + i\x n ,Xi + i \ X n ) 

j=i+2 

< H(Xi\X i+ i\xi,...,Xi) < E H(p t \xi,...,Xi)= E [H(pt)-I(pt;xi,...,Xi)) 

p t £Xi\x i+1 p t eXi\x i+1 

i 
Pt£Xi\X i+1 3=1 

Hence we can write 

n n—ln 

J2 1{X 3 -+ (X, \ X^ Xj ) < E E W \ X i+ i;xj\X i+1 \ X,) 

3=1 i=l j=i-\-l 

n—1 I i \ n 

^E E H( Pt )-\x i \x i + l \Y J H(.x j )\= Y H( Pi )-j2\Xjmxj) 

t=l \p t £Xi\X i+1 3=1 / Pl £X 1 \{Po} 3=1 

n n 

= J2 H(X 1 ^ P.) - E \ X i\ H (*i) < E E H & "> Pi) - E IXiWxj) 

Pi€Xi\{p } j=l Pi€X!\{po} PltXi j=l 

n n 

= E E tffa -* ») - E iXiWxj) < E c ' - E i^-i^fe) 

Piex iPl eA- 1 \{p } J=i Pie* 3=1 



Last inequality is due to property Hi. 
Proof of Lemma 5: 

A j > N, * 

n n I 

J2\Xi\H(xi) -J2 [l(Xi^(X 1 \Xi);xi)+ Y HXt-tpuxiW-tiXiXX!)) 

1=3 1=3 V P' GA "' 

n 3—1 

>Y^\xi\L t +Y, L i- E c - 

1=3 1=1 PiGXj 

Then, all we have to prove is , putting H(xi) in place of Li, 

J — 1 n n 

Y a>E i? ( a; ') + E / ( X '^( X i\^);^) + E Y Ii.Xi^p i ;xi\X l ^{X 1 \X l )) 
Pl exj i=i i=j i=j Pi GX, 

Following lemma will help us 

Sublemma 1. For each j,m > j and variables A,B 



Yl(A ]Xl \Xi -> Xi\X u B) < H(A). 



1=3 



Proof: 



YHA;xi\X l ^X 1 \X l ,B) 



1=3 



Y H(A\Xi -► X, \ X U B) - Y H(A\xi,X t ->. X t \ X U B) 

1=3 1=3 

III 

= H(A\Xj -> X x \ X jt B) + Y H ( A \ X i -> *i \ *i, B) 

l= 3 + l 

771—1 

- J] H(A\x l ,X l -► X! \ J*Q, B) - fl-(A|x/,X m -* X! \ X m , B) 

m— 1 

= H(A\Xj -^X 1 \X J ,B)+Y H{A\X l+1 -► Xi \ X m ,B) 

771—1 

- £ #L4|a;*,X -► Xi \ Xi,B) - H(A\x u X m -► X x \ X m , B) 



1=3 

m—X 

< H(A) + Y [H(A\X l+1 ->- X t \ X l+1 ,B) - H(A\x h Xi -► X x \ X h B)\. 

1=3 

Let us examine Xi — > X\ \ Xi : 

X 1 ^X 1 \X,= (Xi -► X x \ Xi) U (^ \ Xi -> X x \ X t ). 

With a similar idea used in the proof of Theorem 2, we can claim that X\ \ X\ — > X\ \ Xi is completely 
dependent on Xi — ► X\ \ Xi. Thus Xi — > X\ \ Xi is also completely dependent on X{ — > X\ \ Xi. Since 



Xi — > X\ \ Xi C X\ — > X\ \ Xi, reverse is also true. Hence, we can use sets Xi —> X\ \ X\ and X\ — > X\ \ Xi 
interchangeably in information theoretic expressions. Then, 

in 
Y^I^x^ ^ X^XuB) < 

1=3 

m—1 

H(A) +J2iH(A\X l+1 -► X x \X l+1 ,B) - H(A\x h X l -► X^X^B)} = 

1=3 
m—1 

H(A) + Y, [H(A\Xi -► Xi \ X l+1 ,B) - H(A\x l ,X 1 -> X x \ X,,B)] = 

1=3 

771—1 

H{A) + J2 [H(A\X X -> X, \ Jfj+i, Xx -4 Xx \ X h B) - H{A\x h X 1 -► X x \ Jf,, B)] = 

m— 1 

H(^) - ^ I(A;X 1 ->. J*Q \ Jfj+ils,,^! -+*i \ JTi,B) < #(^). 

■ 
Now, let us define r(pi) such that p^ S X r r p .\ \ -^r(pi)+i-Then, 

n r(pj) 

53 53 j(jr,->-i> 4 ;»,|x,-^(jri\j:,))= 53 j^/(x,->-p i ;xi|jri->-(Xi\jr,)) = 

1=3 PiGXj PiGA'j J=j 

r(pi) /r(pi)-l 

E E 7 U ((*m\*m+l)^J*)U(X r(p0 ^pO;*l|*l">*l\*I 

Pi£Xj l—j \ m—l 

By the chain rule 

r(p;) jr(j>i)-l 

Pi£Xj l=j V m— / 

(r(pi)-l m r(pi) 

by Sublemma 1( we also seperate p\ G X n from other peers, and writing it in close form) 

n /r(pi)-l 

<Y / I(X l ^ Pl ;x l \X l ^X 1 \X l )+ J2 Y, H(X m \X m+1 -> Pi )+H(X r(pi) ^ Pi ) 

1=3 Pi£Xj\{ Pl } \ m=j 

Then 

j — 1 n n 

53if(x;) + 53/(X,^(X 1 \X,);x i ) + E^ 53 J(.Xi->i»i;a?i|.Xj-)-(*i\.Xj))< 
53 #(z;) + J] * (*i -»• (*i \ Xi);x t ) + 53 7 W "+ Pi I si|*i -► *i \ *«)+ 

/r(p<)-l 

5^ 53 ^(^m \ ^m+l -> PO + H(X r(pi) -> K ) 

PiGXj\{pi} V m=j 



J-l n 

J2 H{xi) + Y^lHXi -»• (*i \ *0; xi) + I{Xi -> pi; s,|JC, -> X! \ X,)]- 

/r(pi)-l 

E X! ^(^ \ Xm +! "^ P*) + H ( X r(pi) -> Pt 



j — 1 n 

53^(»0+53-f(^i ^Xi\X,X ; ->-pi;xi)+ 
1=1 J=j 

/r(pi)-l 

J2 E H ( Xm \ Xm + 1 -> P ^ + H ( X r(pi) ~> Pi) 

PieXj\{pi} \ ™=J 



j-l n fr(pi-l 

J2 H( Xj ) + ]T Jf (S,) + E E H ( X ™ \ X ™+! "> P*) + #(*r(pO "► Pi) ] = 

'=1 1=3 P;6X,\{pi} \ m=j 



n fr{pi)-l 

Y J H{xi)+ Y. E H(X m \X m+l ^ Pi )+H(X r(pi) ^ Pi ) 

1=1 p<ex 3 \{ Pl } \ m=j 

/r( Pi )-l \ 

H(X 1 -> pi) + £ ]T #pf m \ X m+1 -> p. t ) + F(X r(pi) -> ft ) < 

p«exA{ P i} \ m =J / 

(r{pi)-l \ 

H(X i ^p 1 ) + F(Xi\X i -^pi)+ ^ ]T tf(X m \ X m+1 -> Pi ) + ff(X r(p0 -► Pi ) < 

Pie^\{pi} \ m=j J 

J2 H( Pl -> pO + £ H(p, -4 Xi \ X,) + ^ J2 H & -> Pi) ^ 

PiGXj pi£Xj Pi&Xj PiGXj\{pi} 

X] E h (pi^p>) + E E H (pi^p*) = E E H (pi^Pi) < E c < 

pieXjpieXiVXj PieXjpteXj pi&XjPieXx pi&Xj 

Last inequality is due to property Hi. ■ 

Proof of Lemma 6: 

Let Ai = {iVijiVb, • • • ,iV m _i} and ji = min{r : iV r = ma:r(.Ai)}. Then, iteratively define A, = A,_i \ 
{JVj 4 _i i Nj i _ 1+ i, . . . , iV m _i} and ji = min{r : N r = max(Ai)} for i > 1. Note that 3 / G {1, 2, . . . , m — 1} such 
that ji = 1 since \Aj\ is strictly decreasing and so is the sequence {ji}. Moreover, note that m > j\ > J2 > ■■ ■ > 

jl = 1 and > N jl > N h > . . . > N jt = N r . 

Sublemma 2. 

N- Nl > ^ N 



Xm\ — 1 



Proof: Using £ C t + C s - £ (X^ = -JVi, we get 

N < JV A , 



|-X"m| — i |-X"m| — i 

/l-l 



- ^ l^-l-l \X M \-1 



1 JVi, - TV,:. 



IX,;, I - 1 



since it, — M and N;.,, = 



< ]T Q + a-^ix,-^ 

= -iVi 
Hence, we get tv-^ — t < —Ni which implies N — TVi > ,L , , , as desired. ■ 

Identity 1. 7/j < i, 

^ Ci = N t -Nj + ^L;(pQ|-l) 

PtEX^Xt i=J 

holds. 

Proof: Note that we have J2 Pt£ x 3 C t = EIL, \*i\U + £i=i ii - C. - 2V,-, E p , e x t 3 = £IU IW + 
E'=i L « -C.-Nt and £ Pi6XjAXt C< = Ergjc, C ' ~ E P! ex t d- Hence < we 8 et 

n j—1 n t— 1 

^ q, = (5^|x 4 |ii i +53x 4 -c.-iv J -)-(£;i- x 'ii L <+Z) li i- c '.- JV "*) 

Pi£Xj\Xt i=j i=1 i=t i=1 

t-1 
= JV t - iVj + 5^^(1^1-1). 
*=j 
This completes the proof of identity. ■ 

We will give the proof of lemma in the following two complementary cases: 

Case 1: }* M } N , < N - N h holds. 

\Xm\-Jl, ~ J1 



By Identity Q] we have 



Y, C t = N m - N h + J2 L *(\*i\ ~ 1) 
p i ex il \x m i=h 

> N-N h (16) 

\X M \N 



~ \X M \ - 1 
Hence, Vp. t e X jl \ X m 30 < Cm < C, such that 

\Xm\N 

2^ CiM - Tv—\ — r (17) 

„ rt ,v |A M |-1 

Note that 

V CW < V C m = ,^ |JV . <N-N h <N- Nj (18) 



^V - N h+1 > ' , . >N- N H (20) 



\/j G {ji,ji + 1, . . . ,m — 1} since iV^ > iVj Vj £ {ji, ji +l,...,m— l}by the choice of j\. Now, we choose 
CW = 0, Mpi eli\ X^. Hence, usign CW = 0, Mpi eli\ X^ we get, 

53 c^= 53 aM = ^^ I <^-%<iv-iv j d9) 

Pi ex j \x m Pl ex it \x m ' M 

Vj G {1, 2, . . . , j! — 1} since Nj 1 > Nj Vj G {1, 2, ... , j\ — 1} by the choice of j 1 . Hence, as can be seen from 
d 17t . ( fT8l ) and ( fT9l ), this particular choice of CW Vpi G Xi\X m is exactly as desired. This completes proof of Case 1. 

Case 2 : l^ffK >N- N h holds. 
Substituting iVi = Nj t in Sublemma, we get: 

N-N h >^L>N-N h 

\-&M\ - 1 

which implies that Eli G {1, 2, . . . , 1} such that 

\X M \N 

\X m \ - 1 

holds. Now, we will give an iterative method for the choice of CW, Vpi Gli\ X m satisfying required constraints 
as follows: 

First Step: Choice of C'mj, Wpi G Xj t \ X m 

By dl6l), we have J2 Pi ex \x m &%> N - N 3l . Hence, Vpt G Xj x \ X m 3CW with < C iM < Ci such that 

53 C iM =N-N h . (21) 

Note that 

51 CW < Y, C ™ =N-N jl <N- N (22) 

P i&Xj\x m Pi ex H \x m 

Y? S {ji,ji + 1, . . . , to — 1} since ./V^ > A^ Vj G {ji, Ji +l,...,m— l}by the choice of ji. 
r-f/z step for 1 < r < t : Choice of Cim, Vk G Xj r \ Xj rl . 
By Identity [T] we have 

jr-l 

53 Ci = N jr _, - N JT + J2 Li{\xi\ - 1) 

Hence, Vpi G X,- r \ Xj r _j 3C iM with < Ca/ < Q such that 

53 C iM = N jr _ ± - N jr . (23) 

Using (EJ and d23}, Vr G {2, 3, . . . , t} and Vj G {j r , > + 1, . . . , j r -i + 1} we have 

PieXj\x m p,ex 3r \x m 



53 C'a/+53( 5Z C ' iM ) 



= (N-N h ) + J2W^i- 


-N jq ) 


= N-N jr 




< N-N 3 





(24) 



since Nj r = max{N\, N2, . . ■ , Nj rl+ i} by the choice of j r . 



(t + l)-th step: Choice of CW, Vpi G Xjt+i \ Xj f an ^ ^Pi 6^i\ Xjt+i> separately. 
Remember we have (|20t by the choice of t. Using Identity [T] and d20l ), we get 

£ C *M > N jt -N jt+1 

= (N - N jt+1 ) - (N - N H ) 

Hence \fp t G X jt+1 \ X H , 3C iM with < C iM < Q such that 

P ,ex Jt+1 \x Jt 
Using ( |25T l, we get 

/ „ CiM = 2^ CW + ^ CW 

P.eA' 3t + 1 \X m Pi€X u+1 \X Jt Pi £Xj t \X m 

\X M \N , , , 

= ,' | ; ~ (N - N jt ) + (N - N jt ) 

\Xm\ - J- 

I*m|jv 
= W^-i (26) 

Now, choose Cjjvf = 0, Vpj G -X"i \ -Xj t+i . Using d26b and the choice of C^/ = 0, Vpj G X\ \ Xj t+X , we get for 
any j G {1,2, . . . , j t+1 - 1} that 

2 , c«m = 2^ cw + 2^ Qm 

Pi ex,\* m p.6^\x ](+1 P,ex h+1 \x m 

\X M \N 







|X M |-1 



(27) 



|*m|-i 

Using (|27T i. we get for any j G {1,2,... , j t — 1} that 

2 y ^W < 2^ cw 

P.6^\^ Pi£-X- it+1 \X m 

|x M |-i 

< N-Njt+i 

< N -Nj (28) 

since iVj t+1 = maa;{iVi, N 2 , ■ ■ ■ , Nj t _±} by the choice of j't+i. Putting j = 1 in j27l ) gives 

v |X M |iV 

2- CiM = ^^-I (29) 

Finally, considering d22l i. d24l i and d28l l. this particular choice of Qm for each peer pi G Xi \ X m satisfies 

X! ^ <N-Nj 
p i ex j \x m 

Vj G {1, 2, . . . , m - 1} and by ((29) 

^ |X M |iV 

as desired. This completes the proof of Lemma [6] ■ 
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